TITLE OF THE INVENTION 



CAMERA CONTROL SYSTEM, IMAGE PICK-UP SERVER, CLIENT, 
CONTROL METHOD AND STORAGE MEDIUM THEREFOR 

BACKGROUND OF THE INVENTION 

This invention relates to a camera control system, 
which is capable of controlling a remote camera, ideal 
for application to surveillance cameras, etc. 

A camera control system for controlling the 
shooting direction and zoom magnification of a remote 
camera via a network has already been proposed by the 
assignee of the present applicant. 

However, there is the danger that an imaging area 
in which problems arise in terms of privacy may fall 
within the field of view of the camera. The camera 
cannot be installed at such a location for this reason. 

SUMMARY OF THE INVENTION 

Accordingly, an object of the present invention is 
to provide a camera control system which will not invade 
one's privacy even at a location where there is a 
problem in terms of privacy. 

According to the present invention, the foregoing 



object is attained by providing a camera control 
apparatus for controlling shooting direction and zoom 
magnification of a camera, comprising: input means for 
inputting a control command for controlling shooting 
5 direction or zoom magnification of the camera; 

calculation means for calculating, on the basis of the 
control command input by the input means, an area to be 
= imaged by the camera; limiting means for limiting a 

~ control range of the zoom magnification of the camera in 

H 1 10 a case where the imaging area of the camera calculated 

4= by the calculation means overlaps a prescribed area; and 

O control means for controlling the zoom magnification of 

O the camera so as to fall within the control range, which 

fy has been limited by the limiting means, in a case where 

15 the zoom magnification of the camera exceeds the control 

. -JFi 

range limited by the limiting means. 

In accordance with the present invention, the 
camera control apparatus further comprises storage means 
for storing limit information, which concerns the 
20 control range of the zoom magnification of the camera, 
corresponding to the prescribed area. 

In accordance with the present invention, the 
limiting means limits an upper limit of the zoom 
magnification of the camera in such a manner that the 
2 5 upper limit of the zoom magnification of the camera will 
be lower than in a case where the imaging area of the 
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camera does not overlap the prescribed area. 

In accordance with the present invention, the 
control means controls the zoom magnification of the 
camera to the upper limit of the control range, which 
has been limited by the limiting means, in a case where 
the zoom magnification of the camera has exceeded the 
upper limit of the control range limited by the limiting 
means . 

In accordance with the present invention, there is 
provided a camera control system. for controlling 
shooting direction and zoom magnification of a camera 
from an operation terminal via a network, wherein the 
operation terminal comprises : input means for inputting 
a control command for controlling shooting direction or 
zoom magnification of the camera, and output means for 
outputting the control command, which has been input by 
the input means, to a camera control apparatus that 
controls the camera; and the camera control apparatus 
comprises: calculation means for calculating, on the 
basis of the control command output by the operation 
terminal, an area to be imaged by the camera; limiting 
means for limiting a control range of the zoom 
magnification of the camera in a case where the imaging 
area of the camera calculated by the calculation means 
overlaps a prescribed area; and control means for 
controlling the zoom magnification of the camera so as 



to fall within the control range, which has been limited 
by the limiting means, in a case where the zoom 
magnification of the camera exceeds the control range 
limited by the limiting means. 

In accordance with the present invention, the 
camera control apparatus further comprises storage means 
for storing limit information concerning the control 
range of the zoom magnification of the camera 
corresponding to the prescribed area . 

In accordance with the present invention, the 
limiting means limits an upper limit of the zoom 
magnification of the camera. 

In accordance with the present invention, the 
control means controls the zoom magnification to the 
upper limit of the control range, which has been limited 
by the limiting means, in a case where the zoom 
magnification of the camera has exceeded the upper limit 
of the control range limited by the limiting means. 

The present invention further provides a camera 
control method for controlling shooting direction and 
zoom magnification of a camera, comprising: an input 
step of inputting a control command for controlling 
shooting direction or zoom magnification of the camera; 
a calculation step of calculating, on the basis of the 
control command input by the input step, an area to be 
imaged by the camera; a limiting step of limiting a 



control range of the zoom magnification of the camera in 
a case where the imaging area of the camera calculated 
by the calculation step overlaps a prescribed area; and 
a control step of controlling the zoom magnification of 
5 the camera so as to fall within the control range, which 
has been limited by the limiting step, in a case where 
the zoom magnification of the camera exceeds the control 
range limited by the limiting step. 

^ In accordance with the present invention, the 

H 5 10 camera control method further comprises a storage step 

of storing limit information concerning the control 

Q range of the zoom magnification of the camera 

O corresponding to the prescribed area. 

fU In accordance with the present invention, the 

Hj 

ijg 15 limiting step limits an upper limit of the zoom 

magnification of the camera. 

In accordance with the present invention, the 
control step controls the zoom magnification of the 
camera to the upper limit of the control range, which 
20 has been limited by the limiting step, in a case where 
the zoom magnification of the camera has exceeded the 
upper limit of the control range limited by the limiting 
step. 

A storage medium storing a camera control program 
25 for controlling shooting direction and zoom 

magnification of a camera, wherein the stored program 
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calculates, on the basis of a control command for 
controlling shooting direction or zoom magnification of 
the camera, an area to be imaged by the camera; limits a 
control range of the zoom magnification of the camera in 
5 a case where the calculated imaging area of the camera 
overlaps a prescribed area; and controls the zoom 
magnification of the camera so as to fall within the 
limited control range in a case where the limited 
"12 control range is exceeded. 

™ 10 In accordance with the present invention, the 

HF stored control program further stores limit information 

a 

O concerning the control range of the zoom magnification 

Q of the camera corresponding to the prescribed area, 
fy In accordance with the present invention, the 

ri a 

15 stored control program further limits an upper limit of 

the zoom magnification of the camera. 

In accordance with the present invention, the 

stored control program further controls the zoom 

magnification of the camera to the upper limit of the 
20 control range in a case where the upper limit of the 

limited control range is exceeded. 

In accordance with the present invention, there is 

provided a camera control apparatus for controlling zoom 

magnification of a camera, comprising: input means for 
25 inputting a control command for controlling zoom 

magnification of the camera; calculation means for 
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calculating, on the basis of the control command input 
by the input means, an area to be imaged by the camera; 
limiting means for limiting a control range of the zoom 
magnification of the camera in a case where the imaging 
5 area of the camera calculated by the calculation means 
overlaps a prescribed area; and control means for 
controlling the zoom magnification of the camera so as 
to fall within the control range, which has been limited 
^ by the limiting means, in a case where the zoom 

'rf 10 magnification of the camera exceeds the control range 

i limited by the limiting means. 

E3 The present invention further provides a camera 

Q control method for controlling zoom magnification of a 

ftp camera, comprising : an input step of inputting a 

;yg 15 control command for controlling zoom magnification of 

. 

the camera; a calculation step of calculating, on the 
basis of the control command input by the input step, an 
area to be imaged by the camera; a limiting^ step of 
limiting a control range of the zoom magnification of 

20 the camera in a case where the imaging area of the 

camera calculated by the calculation step overlaps a 
prescribed area; and a control step of controlling the 
zoom magnification of the camera so as to fall within 
the control range, which has been limited by the 

25 limiting step, in a case where the zoom magnification of 
the camera exceeds the control range limited by the 
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limiting step. 

In accordance with the present invention, the 
network is a LAN or ISDN line. 

Another object of the present invention is to 
5 provide an image pick-up server, a client, a control 
method and a storage medium therefor, in which it is 
possible to provide an excellent operating environment, 
q without giving a client an odd impression, while setting 

j5 a plurality of scattered imaging control areas within a 

rf 10 range in which imaging can be performed by image sensing 

2? means whose shooting direction can be changed. 

According to the present invention, there is 
E3 provided an image pick-up server, which has image 

FU sensing means, for performing a distribution service 

%Q 15 which distributes video information obtained by the 

image sensing means to a client via a network, 
comprising: storage means for setting a plurality of 
shootable areas, which are the object of the 
distribution service, within a range in which shooting 
2 0 direction of the image sensing means can be changed, and 
storing information relating to the plurality of 
shootable areas set and information which specifies a 
plurality of virtual image sensing means for respective 
ones of the shootable areas; and control means for 
2 5 reading out, from the storage means, information 

corresponding to virtual image sensing means selectively 
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designated by the client, setting a range in which it is 
possible to change the shooting direction of the image 
sensing means based upon the information relating to the 
shootable area indicated by the information that has 
5 been read out, and controlling the image sensing means 
within the range in which it is possible to change the 
shooting direction . 

^ Other features and advantages of the present 

invention will be apparent from the following 

rf 10 description taken in conjunction with the accompanying 

drawings, in which like reference characters designate 

E3 the same or similar parts throughout the figures 

O thereof . 

=3 S 

jj La 
= !_5 

yQ 15 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of a camera control 
apparatus according to an embodiment of the present 
invention; 

20 Fig. 2 is a diagram showing an example of a look-up 

table serving as zoom limiting information; 

Fig. 3 is a processing flowchart illustrating the 
operation of the embodiment; 

Fig. 4 is a conceptual view showing an example of 
25 the relationship between the imaging area of a camera 

and areas in which zoom magnification is to be limited; 
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Fig. 5 is a conceptual view showing an example of 
the relationship between the imaging area of a camera 
and areas in which zoom magnification is to be limited; 
Fig. 6 is a conceptual view showing an example of 
5 the relationship between the imaging area of a camera 
and areas in which zoom magnification is to be limited; 
Fig. 7 is a conceptual view showing an example of 
q the relationship between the imaging area of a camera 

5 and areas in which zoom magnification is to be limited; 

r? 10 Fig. 8 is a flowchart illustrating the operation of 

2C the embodiment; 

^ Fig. 9 is a block diagram of a camera control 

y system according to this embodiment of the present 

[U invention; 

%S 15 Fig. 10 is a block diagram showing a camera system 

according to a second embodiment of the present 
invention; 

Fig. 11 is a diagram showing a user interface on 
the client side according to the second embodiment; 
2 0 Fig. 12 is a diagram showing an operating sequence 

in the system according to the second embodiment; 

Fig. 13 is a flowchart showing the operation of a 
camera service according to the second embodiment; 

Fig. 14 is a flowchart showing the operation of a 
2 5 camera service according to the second embodiment; 

Fig. 15 is a flowchart showing the operation of a 
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camera service according to the second embodiment; 

Fig. 16 is a flowchart showing the operation of a 
client according to the second embodiment; 

Fig. 17 is a diagram showing the format of each 
command in the second embodiment; 

Fig. 18 is a diagram showing an example of setting 
of a shootable area in the second embodiment; 

Fig. 19 is a diagram showing the structure of a 
camera-control limits list according to the second 
embodiment ; 

Fig. 20 is a diagram showing the configuration of a 
system according to a modification of the second 
embodiment ; and 

Fig. 21 is a diagram showing a camera list in the 
system of Fig. 20. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Embodiments of the present invention will now be 
described with reference to the drawings. 
(First Embodiment) 

Fig. 1 is a block diagram showing a camera control 
apparatus 101 according to this embodiment. A camera 
111 equipped with a pan head is panned and tilted by 
controlling the pan head, thereby controlling the 
shooting direction of the camera. The camera 111 is 
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equipped with a zoom lens the imaging angle of view of 
which can be changed. The image picked up by the camera 
111 is output to a display controller 116. The latter 
executes predetermined signal processing and then 
5 displays the captured image on a monitor 117 . The 

monitor 117 is constituted by a bitmap display or the 
like. 

A camera operating panel 114, which comprises a 
mouse or keyboard (not shown), etc., is capable of 

10 inputting commands for controlling panning tilting and 
zooming of the camera 111. A zoom limiting processor 
113 determines whether to limit zoom magnification in 
dependence upon the pan, tilt and zoom control commands 
entered from the camera operating panel 114. A storage 

15 unit 115, which comprises a RAM or the like, stores a 
look-up table serving as zoom limiting information, 
which is shown in Fig. 2. The look-up table is composed 
of the following for each limited area: x coordinates 
(coordinates in the pan direction) , y coordinates 

20 (coordinates in the tilt direction) and the maximum 

allowable zoom ratio for the area. The coordinates of 
the center of a range in which the panning and tilting 
of the camera 111 can be changed is assumed to be (0,0) . 
The zoom limiting processor 113 decides limiting of 

25 zoom magnification based upon the look-up table shown in 
Fig. 2. A camera pan-head controller 112 controls 
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panning tilting and zooming of the camera 111 based upon 
the commands input from the camera operating panel 114 
for controlling panning tilting and zooming of the 
camera 111, and a zoom limiting command from the zoom 
5 limiting processor 113. In this embodiment, the camera 
pan-head controller 112 and zoom limiting processor 113 
may be implemented by processing executed by the same 
microprocessor, such as a CPU. 

Fig. 3 is a flowchart of processing illustrating 
!*? 10 operation of this embodiment. Figs. 4 through 7 are 

J: conceptual views showing an example of the relationship 

O between the imaging area of the camera 111 and areas in 

O which imaging is to be limited. An area 2 01 indicates 

TU the maximum control range over which imaging is possible 

.<yg 15 by panning and tilting a camera. Numerals 202 and 203 

denote areas in which zoom magnification is to be 
limited. As illustrated also in Fig. 2, the limitations 
on zoom magnification are such that the magnifications 
are 2 . OX and 1 . 5X in areas 2 02 and 203, respectively. 
20 An area 2 04 indicates an imaging range calculated at a 
step S3 02, described later. It should be noted that if 
zoom magnification is not limited, then the zoom 
magnification of the camera 111 is capable of being 
controlled up to a maximum of 10X. 
25 First, at step S301 in Fig. 3, a command for 

controlling the zoom magnification of the camera 111 or 
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commands for controlling panning and tilting of the 
camera 111 are input from the camera operating panel 
114. Control then proceeds to step S302, at which an 
area in which imaging is be performed by the camera 111 . 
5 is calculated in advance by the zoom limiting processor 
113 based upon the entered commands. 

If it is found at step S3 03 that the imaging area 
q 204 calculated beforehand overlaps the area 202 limited 

to the zoom magnification shown in the look-up table of 
^ 10 Fig. 2, control proceeds to step S3 04. Fig. 4 

J: illustrates an example of the relationship between the 

^ calculated imaging area and the limited areas at this 

£3 time. If the calculated area does not overlap the areas 

[U of limited zoom magnification, control proceeds to step 

in s 

ID 15 S3 06. 

It is determined at step S3 04 whether a requested 
zoom magnification is greater than the maximum allowable 
zoom magnification (2. OX) of the limited area 202. If 
the requested zoom magnification exceeds the limit on 

2 0 the zoom magnification set for the limited area 202, 

control proceeds to step S305. Here the camera pan-head 
controller 112 revises the control target of zoom 
magnification of camera 111 from the usual controllable 
zoom magnification (10X) to the maximum allowable zoom 

25 magnification (2. OX) of limited area 202. On the other 
hand, if it is found at step S304 that controlled zoom 
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magnification does not exceed the limit on zoom 
magnification (in which case the relationship between 
the imaging area and the limited area at this time will 
be as shown in Fig. 7), then control proceeds to step 
5 S3 06. If the check regarding the area 2 02 of limited 
zoom magnification is finished, then, in order to check 
the area 203 of limited zoom magnification, the 
q processing of steps S3 03 to S3 0 5 is executed. By way of 

jg example, if the calculated imaging area 204 and the 

y[ 10 limited area 2 03 overlap, as shown in Fig. 5, owing to 

2 revision of the control target of zoom magnification to 

" 2 . OX by the above-described processing, then the revised 

y zoom magnification exceeds the maximum allowable zoom 

RJ magnification of limited area 203. Consequently, the 

yQ 15 control target of zoom magnification is revised again, 

becoming 1.5X. The relationship between the imaging 
area and the limited areas in this case is as shown in 
Fig. 6. 

If the checking of all areas of limited zoom 
20 magnification present in the look-up table is completed, 
control proceeds to step S307. 

If it is found at step S3 07 that the control target 
of zoom magnification has been revised, control returns 
to step S302 so that all limited areas in the look-up 
25 table are checked again through steps S302 to S306. If 
it is determined by this recheck that the revision of 
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the zoom magnification is correct, control proceeds to 
step S3 08, where a control signal for controlling the 
camera 111 to the revised zoom magnification is output 
to the camera pan-head controller 112. On the basis of 
5 this control signal, the camera pan-head controller 112 
drives the zoom lens of the camera 111 and, on the basis 
of pan and tilt control commands from the camera 
« operating panel 114, controls panning and tilting to 

move the shooting direction. 
10 In accordance with this embodiment, as described 

jpEj above, controlling the panning and tilting of the camera 

^ 111 provides a limit on zoom magnification in an area, 

O in which privacy is a concern, located in the range 

PJ capable of being shot by the camera. An area in which 

=jy 15 imaging will take place is calculated beforehand based 

upon a control command input by the camera operating 
panel 114. If overlap occurs between the calculated 
imaging area and an area in which zoom magnification is 
limited, reference is had to the allowable zoom 
20 magnification of the limited area and a limitation is 
imposed upon zoom control in such a manner that the 
upper limit of zoom magnification is lowered below that 
which prevails when there is no overlap. As a result, 
it is possible to control imaging by the camera, even at 
25 locations where privacy is a concern, without infringing 
upon such privacy. Since the maximum allowable zoom 
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magnification can be made different for each limited 
area, an apparatus can be provided that makes it 
possible to deal flexibly with limited areas having a 
high degree of importance. 
5 In the embodiment described above, the zoom 

magnification is limited if overlap occurs between the 
imaging area calculated by the zoom limiting processor 
r=s 113 and the area in which zoom magnification is to be 

"% limited. However, an arrangement may be adopted in 

10 which zoom magnification is limited in a case where the 
J; position of the center of a calculated imaging field of 

3 view falls within the area in which zoom magnification 

O is to be limited. 

HJ Further, as shown in Fig. 9, this embodiment can be 

%g 15 applied to a camera control system in which it is 

possible to control the camera control apparatus 101 
from a remote operating terminal 102 via a network 103 
such as a LAN (Local Area Network) or ISDN (Integrated 
Services Digital Network) line. A plurality of camera 
20 control apparatus and operating terminals 102 can be 
connected on the network 103 so that a desired camera 
control apparatus 101 can be controlled from any control 
terminal 102. Blocks shown in Fig. 9 that are identical 
with those of Fig. 1 are designated by like reference 
25 characters and need not be described again. 

The operating terminal 102 includes a controller 
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125 for controlling the overall operating terminal 102, 
and a camera operating unit 124, which comprises a mouse 
and keyboard, for controlling panning, tilting and 
zooming of the camera 111. A control command that has 
5 been entered by the camera operating unit 124 is output 
to the camera control apparatus 101 via a communication 
interface 121. The camera control apparatus 101 has a 
communication interface 119 for receiving pan, tilt and 
zoom camera control signals output by the operating 

10 terminal 102 and outputting these signals to the zoom 

limiting processor 113, and the zoom limiting processor 
113 executes processing similar to that described above. 
The camera control apparatus 101 further includes a 
video processor 118 for applying compression processing 

15 to an image signal picked up by the camera 111, 

converting the signal to a signal conforming to the 
network specifications, and outputting the signal from 
the communication interface 119 to the operating 
terminal 102 . The operating terminal 102 further 

20 includes an image expansion unit 122 for expanding image 
data received at the communication interface 121, and a 
display controller 125 for executing prescribed signal 
processing and causing a monitor 126 to display a sensed 
image . 

25 The above-described embodiment uses a camera, which 

is equipped with a pan head, the shooting direction of 
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which can be controlled. However, the processing of 
this embodiment can be executed even with a camera whose 
shooting direction is fixed. Fig. 8 is a flowchart of 
processing for operating the camera 111 in a case where 
5 the shooting direction is fixed. Except for the fact 
that input of pan and tilt control commands is excluded 
from step S801 and execution of pan and tilt control is 
q excluded from step S808, this processing is identical 

% with that shown in Fig. 3 and need not, therefore, be 

2 10 described again. 

J: In this embodiment, input means corresponds to the 

~ camera operating panel 114 or camera operating unit 124, 

calculation means corresponds to the zoom limiting 
Rj processor 113, limiting means corresponds to zoom 

£i 15 limiting processor 113 and storage unit 115, control 

means corresponds to the camera pan-head controller 112, 

and output means corresponds to the communication 

interface . 

Thus, in accordance with the first embodiment, as 
20 described above, it is possible to provide a camera 

control system which will not invade one's privacy even 
at a location where there is a problem in terms of 
privacy. 

(Second Embodiment) 
25 A second embodiment of the present invention will 

now be described in detail with reference to the 
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accompanying drawings . 

Briefly stated, the second embodiment is such that 
when a client looks through a camera that has been 
connected to a camera server, it appears just as if 
there are a plurality of cameras. 

Fig. 10 of a system according to the- second 
embodiment . 

The system shown in Fig. 10 includes a camera 
server 1-1, a display control terminal (referred to as a 
"client") 1-2 for displaying video from the camera 
server 1-1 and remotely controlling a camera, and a 
network 1-3 for connecting the camera server 1-1 and 
client 1-2. 

The camera server 1-1 captures video from a video 
camera 1-11, distributes the video to the client 1-2, 
accepts camera control commands from the client 1-2 and 
controls the zoom magnification of the video camera 1-11 
and a movable pan head 1-12. (Operating the movable pan 
head 1-12 makes it possible to change the pan and tilt 
angles . ) 

Though one camera server 1-1 and one client 1-2 are 
illustrated in Fig. 10, this is for the sake of 
simplifying the description and there is no limitation 
upon the numbers thereof . The description given below 
also assumes one camera server and one client. As long 
as the network 1-3 is a digital network such as the 
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Internet or intranet having a bandwidth sufficient for 
transferring a camera control signal and compressed 
video signal, described later, any type of network will 
suffice. 

5 It is assumed that the TCP/IP (UDP/IP) protocol is 

the network protocol in this embodiment, and when an 
address is mentioned below, this is taken to indicate an 
IP address. Further, it is assumed that IP addresses 
have been assigned to the camera server 1-1 and client 
10 1-2. 

The camera server 1-1, which can be implemented by 
a personal computer, includes the video camera 1-11; the 
movable pan head 1-12 for controlling pan and tilt 
angles of the video camera 1-11; a camera /pan-head 

15 controller 1-13 for controlling the video camera 1-11 
and the zoom magnification and pan and tilt angles of 
the movable pan head 1-12; a video input unit 1-15 for 
capturing video from the video camera 1-11; a video 
compression unit 1-16 for compressed captured video 

20 data; a communications control unit 1-14 for 

distributing compressed video data over the network; a 
command interpretation/ execution unit 1-17 for 
interpreting commands from the client 1-2 via the 
network 1-3 and controlling the components of the camera 

2 5 server; and a storage unit 1-18 used in passing data 
among the various components . 
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The video input unit 1-15 captures an NTSC video 
signal from the video camera 1-11, converts the analog 
signal to a digital signal, compresses the digital 
signal in accordance with motion JPEG and delivers the 
5 resulting signal to the communications control unit 
1-14. Though compression according to motion JPEG is 
used here as the video compression format, it is obvious 
•q that this does not impose a limitation and any 

]E compression format by may be employed. 

10 The client 1-2 will now be described. As in the 

j! case of the server, the client can also be implemented 

^ by a general -purpose information processing apparatus 

Q such as a personal computer. Compressed vidieo data 

fy distributed from the camera server 1-1 is received 

J3 15 through a communication control unit 1-21, expanded by a 

video expansion unit 1-2 5 and displayed on a video 
display unit 1-26. It is also arranged so that camera 
control and operations for storing and saving images can 
be performed by operating a user interface with which 
20 the client is provided. Control for display and 
manipulation of screens is performed by a display 
controller 1-24. 

The video display unit 1-2 6 includes a bitmap 
display that is capable of constructing a screen (user 
25 interface) of the kind shown in Fig. 11. A window 

system is used as the video display unit 1-26 in this 
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embodiment. Examples of this system are Microsoft's 
Windows 95 and the UNIX X-Window, etc. 

As shown in Fig. 11, a video display area 2-1 is 
for displaying video, and a camera operation panel 2-2 
is for operating a camera. The latter is displayed in a 
window format. The camera operation panel 2-2 has 
scroll bars 2-22, 2-23, 2-28 that make it possible to 
manipulate camera pan angle, tilt angle and zoom 
magnification. The scroll bar 2-22 for manipulating pan 
angle has a rectangular scroll box, which is displayed 
in the bar, for being moved to the left or right 
(generally referred to as "dragging") by a pointing 
device such as a mouse, whereby a command designating 
the pan angle is sent to the camera server. The 
position at which the scroll box is displayed indicates 
the present position in an area in which manipulation of 
the pan angle is limited. The scroll bar 2-23 is for 
manipulating tilt angle, and the scroll bar 2-28 is for 
manipulating zoom magnification. The method of 
manipulation and the meaning of the positions of the 
scroll boxes are similar to those of the scroll bar 2-22 
for pan angle . 

As described earlier, users of a plurality of 
clients can view video from the same camera of the same 
camera server. In the end, however, only one client has 
the right to control the camera. For other clients, 




therefore, the scroll bars 2-22, 2-23 and 2-28 are 
displayed in a form to show that they are inoperable (as 
by displaying the scroll bars in dimmed fashion) . The 
privilege to control the camera is acquired by using a 
5 mouse or the like to click a control start button 2-21 
shown in Fig. 11. If the button 2-21 is clicked, a 
control-privilege acquisition request is sent to the 
camera server at the destination of the connection. 
When the request is received and another client has 

10 already been issued the control privilege, the camera 
server notifies the requesting client that the request 
is rejected. If no other client has been issued the 
control privilege, then the camera server issues this 
privilege to the requesting client. It should be noted 

15 that the camera server measures the length of time the 
control privilege has been granted to one client and, 
when a predetermined length of time expires, sends this 
client notification of the fact that the control 
privilege will be nullified. 

2 0 With reference again to the user interface of Fig. 

11, a camera changeover button 2-30 is for displaying a 
pull -down menu of the kind shown at 2-31 when the button 
2-30 is clicked. In the case of the pull-down menu 
2-31, any of three cameras denoted camera 1 to camera 3 

25 can be selected. The camera names do not indicate 

actual cameras (the camera server in this embodiment has 
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only one physical camera) but correspond to "virtual 
cameras", described below. In response to camera 
selection, a changeover (selection) is made among the 
virtual cameras. Further, it is assumed that the name 
of the virtual camera currently selected is displayed on 
the button 2-30. List information indicating selectable 
virtual cameras is transmitted from the camera server 
when this client is connected to the camera server. 

A virtual camera will be described with reference 
to Fig. 18. 

A border 9-3 indicates a range within which imaging 
is possible over the full range of pan and tilt angles 
capable of being shot by the camera 1-11 of the camera 

this 

server 1-1. ( ^dabi fc «b is a range in which the zoom 
magnification of the camera is minimum. ) The areas 
represented by the hatching indicate areas in which 
shooting is allowed. In other words, the area outside 
these hatched areas indicates an area in which imaging 
is not allowed for reasons of privacy, etc. 

Since the pan and tilt angles of the camera 1-11 
can be changed, the user of the client would naturally 
think that the shooting direction (angle) of one camera 
could be varied over an area having a rectangular shape. 
In other words, it would seem odd to the user if one 
camera could shoot in an L- shaped area and in another 
area separated from it, as shown in Fig. 18. In order 



not to give the user such an odd impression, therefore, 
this embodiment is so adapted that an area in which 
image pick-up is possible is divided into several 
rectangular areas and virtual cameras are assigned to 
5 respective ones of these rectangular areas (the three 
areas indicated at 9-11, 9-12 and 9-13 in Fig. 18) . As 
a result, as far as. the user is concerned, it appears 
just as if a plurality of (three) cameras (virtual 
cameras) have been connected to the camera server 1-1 

10 and just as if the angles of these cameras can be 

changed freely within the respective rectangular areas. 
The user, therefore, does not receive an odd impression. 
The camera names displayed in the pull-down menu 2-31 of 
Fig. 11 thus indicate virtual cameras. 

15 Assume that the pan angle is represented by 0 (the 

farther the camera is panned to the right, the greater 
the value of 9), that the tilt angle is represented by <|> 
(the farther the camera is panned upward, the greater 
the value of <J)) , and assume that the zoom magnification 

20 is represented by z (the larger the magnification, the 
larger the value of z) . The range within which a 
specific virtual camera can be controlled can be set as 
follows, by way of example: 

0min < 9 < 9max (9min: pan angle, left extreme; 

25 9max: pan angle, right extreme) ; 

<|)min < <() < ((tttiax (<|>min: tilt angle, lower limit; 
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(|)max: tilt angle; upper limit) ; 

Zmin < Z < Zmax (Zmin: zoom magnification, lower 
limit; Zmax: zoom magnification; upper limit) . 
t By way of example, if this is applied to the 

5 rectangular area 9-11 in Fig. 18, the pan and tilt 
control range of the virtual camera corresponding to 
this area will have the following diagonal coordinates: 
Q (9min, ())min) , (Gmax, (|)max) . 

J= Of course, (Gmin, (|>min) , (Gmax, (j)max) fall within 

10 the overall physical range within which control is 
S possible. Though the number of virtual cameras is three 

7* in this embodiment, this does not impose a limitation 

~J? because, in theory, any number of virtual cameras can be 

y=J set up. Further, it does not matter if the areas 

® 15 indicating the ranges within which virtual cameras can 

be controlled overlap. It is assumed that all of the 
virtual cameras have been assigned ID numbers for 
identifying these cameras. In terms of internal 
structure, it is assumed that a camera control-range 
20 table of the kind shown in Fig. 18 is stored in a shared 
memory 3-43. In addition, the number of physical 
cameras connected to one camera server is not limited to 
one; two or more physical cameras may be connected. In 
such case one physical camera and the table shown in 
25 Fig. 19 would be made to correspond. That is, tables of 
the kind shown in Fig. 19 would be provided in a number 
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equivalent to the number of physical cameras. 

Fig. 12 illustrates a process in accordance with 
the present invention. This process signifies one in a 
multitask operating system such as a Windows NT or UNIX 
operating system. It is assumed that all processes 3- 
11, 3-12, 3-21, 3-22 and 3-31 are operating in the 
steady state. 

Each process that operates on the camera server 1-1 
will now be described. 

•Camera control server process 3-11: This process 
executes control for accepting a camera control command, 
which has been issued by a client process 3-31 having 
the control privilege, and outputting the command to the 
camera/pan-head controller 1-13. 

•Camera-status notification process 3-12: This 
process senses the states of pan and tilt angles of a 
camera and notifies the client process 3-31. 

•Video server process 3-21: This process manages 
the destination of camera video transmission. 

•Video acquisition/transmission process 3-22: This 
process captures and transmits camera video. 

It should be noted that a video-display inhibit 
flag 3-41, a transmission destination list 3-42, a 
camera control-range limiting list 3-43 and a camera- 
status register 3-44 have been provided in a shared 
memory used in passing data between processes. Further, 
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a camera- control limited range register 3-45, a control 
privilege flag 3-51 and a camera number list 3-52 are 
registers for temporarily saving data used in the 
processes . 

5 The details of the operation of processes 3-11 to 

3-22 in the camera server 1-1 are illustrated in the 
flowcharts of Figs. 13, 14 and 15. The client process 
q 3-31 runs on the camera client 1-2, and the details of 

this operation are illustrated in the flowchart of Fig. 
rf 10 16. It should be noted that the programs corresponding 

2r to these flowcharts have been stored in a storage device 

" (e.g., a hard disk or the like) possessed by the server 

y or client and are loaded to a main memory (main RAM) in 

IU order to be executed. 

yQ 15 First the processes of the camera server will be 

described with reference to the flowcharts of Figs. 13 
to 15. 

It should be noted that packets having a format of 
the kind shown in Fig. 17 are generated and transmitted 
20 to the network 1-3. (Strictly speaking, use is made of 
a format employed in TCP/IP or UDP/IP packets, etc. In 
Fig. 17, however, only packet information necessary for 
describing this embodiment is shown. ) 

The camera control process and camera-status 
2 5 notification processing will be described using the 
flowchart of Fig. 13. 
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When the camera control server process 3-11 is 
started up (step S400) , the camera-status notification 
process 3-12 is generated (step S402) and a camera- 
control connect request [(1) in Fig. 17] from the client 
5 process 3-31 is awaited (step S403). When there is a 
connect request from the client process 3-31, connect 
acceptance processing (for sending back Ack) is executed 
(step S404) and camera control commands [(2) to (4) in 
Fig. 17) sent from the connected client process 3-31 are 

10 awaited (step S405) . In connect acceptance processing, 
the client is notified of the list information 
concerning the virtual cameras possessed by the camera 
server, as described earlier. That is, the list 
information is transferred to the client. At this time 

15 a timer is set. If a certain fixed time (referred to as 
"control waiting time") elapses, the camera control 
connection is severed (step S407) . 

If camera control commands [(2) to (4) in Fig. 17] 
are received, the camera/pan-head controller 1-13 

20 controls the zoom magnification of the video camera 1-11 
and the pan and tilt angles of the movable pan head 1-12 
in accordance with these commands (step S409) . Until 
control waiting time elapses, camera control commands 
are accepted from the client process 3-31 and camera 

25 control conforming to the commands is performed through 
the command interpretation/execution unit 1-17 and 
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camera /pan-head controller 1-13. The details of 
processing conforming to control commands are 
illustrated in Fig. 14. In order to simplify the 
description, it is assumed that the camera control 
5 commands are as follows: 

pan -angle change command PAN 0 

tilt-angle change command TIL <|> 

p zoom-magnification change command ZOM a 

CP 

jp It should be noted that 9, <|), (X are parameters 

13 

10 representing pan angle, tilt angle and zoom 
S magnification, respectively. Though various settings of 

7* values for a backlighting correction, autofocus and 

*z manual focus are possible, these are not described here. 

The following command [(5) in Fig. 17] for changing 
MP 15 over virtual cameras also is available: 

camera changeover command SEL n 

(where n indicates the identification number of a 
virtual camera) . 

When a camera changeover request command is issued, 
20 a control range corresponding to the identification 
number of a virtual camera is read out of the camera 
control-range limiting list 3-43 and set in the camera- 
control limited range register 3-45 and camera-status 
register 3-44 (step S502) . Next, pan, tilt and zoom 
25 values calculated in the next procedure are initialized 
as the initial position of the virtual camera (step 
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S504) . The initially set values of the virtual camera 
are values at the center of the limited range in regard 
to the pan and tilt angles and minimum magnification of 
the limited range in regard to the zoom magnification. 
However, it is permissible to decide the pan and tilt 
angles and the zoom magnification of the limited range 
in advance and then set the pan and tilt angles and the 
zoom magnification to these values. 

There is a possibility that an area in which a 
video transmission is not allowed will be traversed 
during a changeover of virtual cameras, namely during a 
change in camera panning, tilting and zooming. The 
video-display inhibit flag is set (turned ON), 
therefore, so that video will not be transmitted at such 
time (step S503). When the video-display inhibit flag 
is ON, " -1" information is reported to each client as 
the identification number of the virtual camera, as will 
be described later. Thus, in a case where the selected 
virtual camera is in the state that has been set at step 
S502 (regarding pan angle, tilt angle and zoom 
magnification) , the video-display inhibit flag is turned 
off. As a result, processing for distributing video can 
be executed again. 

In the case of a pan/tilt or zoom request command, 
on the other hand, it is determined whether the pan 
angle, tilt angle and zoom magnification fall within the 




control range of the virtual camera (steps S511, S521) . 
In a case where a value is outside the range, the value 
is reset to the upper limit (in case of a value that is 
too large) or lower limit (in case of a value that is 
5 too small) of the control range. Camera control is then 
carried out . 

If a fixed period of time elapses from the start 
Q of the connection for camera control and control waiting 

j£ time expires, disconnect processing (step S407) is 

£1 10 executed. During the processing of step S407, a camera - 

g control disconnect command [(8) in Fig. 17], i.e., a 

7* command which takes away the camera control privilege, 

if is sent back to the client process 3-31. Arrival of a 

connect request is again awaited (step S403). The 

i y 

"^0 15 camera control server process 3-11 accepts a camera- 

control connect request from any client process 
similarly implemented and operates the camera but, as 
will be understood from Fig. 13, a plurality of client 
processes cannot be connected simultaneously. 

20 Accordingly, it is assumed that this connection is 
connection oriented, as in the manner of TCP/IP. 

The status of the camera is checked continuously 
while the camera-status notification process 3-12 
generated at step S402 is operating. That is, camera 

25 pan and tilt angles and zoom magnification (= p, t, z) 
are read in by making an inquiry of the camera /pan-head 
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controller 1-13, these are written to the camera- status 
register 3-44 (step S422) and the status of the video- 
display inhibit flag 3-41 is checked (step S423) . If 
the flag is ON, it is construed that the camera status 
5 is "CAMERA BEING SWITCHED" (virtual -camera 

identification number n = -1) (step S424) and the client 
process 1-2 that is transmitting video is notified of 

p the camera status [format of (9) in Fig. 17] (step 

2 S425) . 

2 10 The video server process and video 

_E acquisition/transmission process will now be described 

" with reference to the flowchart of Fig. 15. 

^ When the video server process 3-21 is started up 

HJ (step S600) , the video acquisition/ transmission process 

ni 

S 15 3-22 is generated (step S602) and an event from the 

video server process 3-21 is awaited (step S603). If 
the event is a video-display start request command [(7) 
in Fig. 17] (step S604) , then the packet transmission- 
source address contained in the video-display request 

20 packet is checked and added to the video transmission 
destination list 3-42 (step S605) . If the event is a 
video-display stop request command (step S606) , then, in 
similar fashion, the packet transmission-source address 
contained in the video-display request packet is checked 

25 and deleted from the transmission destination list 3-42. 
It should be noted that the address of the video- 



- 34 - 



transmission destination is retained in the video 
transmission destination list 3-42 in a list format. 

When the video acquisition/transmission process 
3-22 is started up, the following operation is repeated: 
The camera video is captured as a frame (step S622) and 
compression is carried out (step S623). If the video- 
display inhibit flag is ON (step S624) , the compressed 
video data is converted to the packet formation shown at 
(10) in Fig. 17 and is transmitted to the plurality of 
addresses in the video transmission destination list 
3-42 (step S625) . 

The client process 3-31, which operates on the 
client side, will now be described in accordance with 
the flowchart of Fig. 16. 

When this process is started up, initialization 
processing is executed (step S701) and a video-display 
start request is transmitted to the camera server 
corresponding to the address (IP address, here assumed 
to be " ADDR_C " ) of the connected camera server 1-1 (step 
S7 02) . The packet format at this time is as indicated 
at (6) in Fig. 17 . 

If Ack is not sent back from the camera server that 
corresponds to ADDR__C (step S703), this means that 
operation is abnormal, e.g., that the address is 
erroneous. The client process, therefore, is terminated 
(step S704) . 



If Ack is sent back, then the display is successful 
(step S703) and the client process waits for an event, 
i.e., a user input from the user interface or reception 
of various packets from the camera server 1-1 (step 
5 S705). 

If the user clicks the control start button 2-21 
(step S7 06) , the user verifies, based upon the control 
O privilege flag 3-51 (a variable reserved in the RAM 

jg possessed by the client) , whether the prevailing state 

10 is one in which camera control has already started (step 
P S707). If camera control is already in progress, 

processing returns to step S705. If control is not in 
^ progress, then the camera-control connect request [(1) 

m in Fig. 17] is sent to the camera control server process 

ffl 15 (step S708) and acknowledgment (Ack) is awaited (step 

S709). If Ack is sent back, a connection is established 
with respect to the camera control server process, the 
control privilege flag 3-51 in the storage unit 1-29 of 
the client apparatus is turned ON (step S711) and 
2 0 operation at the camera operation panel 2-2 (the buttons 
for changing shooting direction and the scroll bars) is 
enabled. It should be noted that the camera control 
server process accepts a connect request only at the 
time of step S403 for camera-control connect request 
25 standby. 

If the connection is established and control 
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waiting time elapses, the camera-control disconnect 
command [(8) in Fig. 17] is sent back from the camera 
control server process. When this is accepted (step 
S714) , the control privilege flag 3-51 is turned OFF 

(step S715) to nullify operation at the camera operation 
panel 2-2 (step S716) . 

During the time that operation at the camera 
operation panel 2-2 is enabled, a camera control command 
that corresponds to the operation performed at the 
camera operation panel 2-2 is generated' and sent to the 
client 1-2 (steps S717 - S718) . In a case where a 
camera has been selected from the camera- changeover 
pull-down menu 2-31 of the camera operation panel 2-2, a 
camera changeover command [(5) in Fig. 17] is issued 

(steps S719 - S72.0) . It should be noted that the 
process through which commands are generated is not 
shown. 

In response to a client-process terminate request 
(step S72 6) issued by operation using the menu or the 
like, the camera-control disconnect command [(8) in Fig. 
17] is issued (step S728) and the client process is 
terminated (step S729) . 

If, in a case where a packet has arrived (step 

5721) , the packet is video data [(10) in Fig. 17] (step 

5722) , compressed video data contained in this video 
data is read out and subjected to expansion processing, 




after which the displayed video on the video display 
panel 2-1 is updated using this video frame data (step 
S723) . 

In a case where the arriving packet is for camera- 
5 status notification [ (9) in Fig. 17] (step S724), the 

display positions of the scroll boxes in the scroll bars 

2- 22, 2-23, 2-28 for manipulating the pan and tilt 
angles and zoom magnification are changed to the 
corresponding positions using parameters contained in 

10 the packet. Further, the camera name corresponding to 
the identification number of the virtual camera is 
displayed on the camera changeover button 2-30. (In a 
case where another client process is remotely 
controlling the camera, this information is updated. ) 

15 The association between the identification number of the 
virtual camera and the camera name makes use of the. 
corresponding relationship of which notification has 
been given by the camera-status notification process 

3- 12. It should be noted that when the identification 
20 number of the virtual camera is -1 (i.e., when the 

camera is currently moving within the shootable range of 
the virtual camera) , it is construed that the operation 
for changing over the virtual camera is currently being 
carried out and a message to this effect is displayed on 
25 the video display panel 2^1. 

In accordance with the second embodiment, the 
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result of the above-described operation is that, when 
viewed from the client, one physical camera possessed by 
a camera server appears just as if it were a plurality 
of cameras (virtual cameras) . Moreover, if at least the 
5 pan and tilt angles of each individual virtual camera 
are within a rectangularly shaped zone, they can be 
changed at will. Consequently, it does not matter that 
Q the user of the client is unaware that an area in which 

jg imaging is limited because of privacy-related reasons 

£1 10 exists within this zone. In addition, the user is not 

S- given an odd impression. 

y As described earlier, it is also possible to 

*2! provide one camera server with a plurality of physical 

LH cameras. In this case also it would suffice to set up a 

-S 15 plurality of virtual cameras for each individual 

physical camera. It would then be so arranged that the 
camera server give the control privilege to the client 
independently on a per-physical-camera basis. 

In the case of the administrator of the client 
20 server, the administrator would operate the camera 
server directly to edit the camera-control limiting 
list. However, an arrangement may be adopted in which 
the administrator is connected in the same manner as the 
client, notifies the camera server of the fact that he 
25 or she is the administrator (e.g., as by giving an ID or 
password) and transfers the file of the limiting list to 
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the camera server by a protocol such as ftp. 

Alternatively, in the case of the administrator, an 
arrangement may be adopted in which the border of a 
range within which the pan and tilt angles of a physical 
5 camera can be changed is displayed on a screen and, 
using a pointing device such as a mouse, the 
administrator sets several rectangles within this border 
p to thereby set virtual cameras corresponding to these 

jg rectangular areas as well as pan and tilt angles over 

lI 10 which these virtual cameras can be moved. The 

g administrator may set upper and lower limit values of 

^ zoom magnification in regard to set rectangular areas 

% using a keyboard or mouse, etc., and processing 

according to the first embodiment described earlier may 

sjj 

B 15 then be executed. Of course, in an instance where the 

administrator has logged in on the client side in order 
to accomplish this, a program for providing the 
administrator with the above-described control need, only 
be run. 

20 Next, a situation in which a camera server having a 

plurality of control limiting areas (a plurality of 
. virtual cameras) and a camera server for which no 
particular restrictions have been set are connected to a 
network in mixed fashion will be described with 
25 reference to Fig. 20. 

In a situation where a plurality of camera servers 
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exist, desirably a user utilizing a client server is 
presented with a list of the camera servers (e.g., a 
list of the kind shown in Fig. 21 is displayed) and 
employs a pointing device such as a mouse to select from 
5 the list the camera server he or she wishes to see. 

When a camera server is selected by the user, the 
camera client transmits the video-display start request, 
Q as at step S702 in Fig. 16. If the present invention is 

utilized, a plurality of control limiting areas can be 
10 handled as separate, independent cameras in the same 
manner as another camera server. For example, in the 
user interface shown in Fig. 21, cameras 1 to 3 
correspond to respective ones of control limiting areas 
of one and the same camera server, and cameras 4 and 5 
15 are separate camera servers. 

The present embodiment is characterized by the 
following: 

1) A camera server is provided with means for 
managing a plurality of camera control limiting areas, 

20 namely areas (referred to as "control limiting areas") 
in which the range of camera pan and tilt angles and 
zoom magnification is limited; 

2) the individual limiting areas are limited in 
control range by setting left and right extremes of pan 

25 angles, upper and lower limits of tilt angles and upper 
and lower limits of zoom magnification; 
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3) it appears to the client process as if each 
control limiting area is a separate, independent virtual 
camera ; and 

4) in a case where a plurality of control limiting 
areas are switched among, it appears to the client 
process as if a plurality of cameras are being switched 
among. 

As a result of the above, it is possible to set 
camera-control limiting ranges more realistically and 
flexibly for areas involving privacy-related problems. 
In other words, camera-control limiting areas are set 
simply one at a time. A plurality of such control 
limiting areas can be set based on the concept of a 
plurality of virtual cameras. The person setting a 
camera control range, therefore, need only set the 
control limiting ranges as different cameras. From the 
standpoint of the client process, these areas seem to be 
separate cameras having different control limiting 
ranges. This makes it easy to grasp the limiting 
ranges . 

According to the first and second embodiments, the 
camera server requires hardware, namely image sensing 
means and means for connecting it to a network. 
However, this can be realized by a general -purpose 
information processing apparatus such as a personal 
computer. Further, as the client merely lacks the image 
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sensing means, the basic hardware thereof is the same as 
that of the camera server and it, too, can be 
constructed using a general -purpose information 
processing apparatus such as a personal computer. 
5 In other words, it will suffice if the necessary- 

hardware is exploited and a program for operating the 
apparatus as a camera server or client is provided. 
q Accordingly, the object of the invention is attained by 

% supplying a storage medium storing the program codes of 

rf 10 the software for performing the functions of the 

2l foregoing embodiments to a system or an apparatus, 

^ reading the program codes with a computer (e.g., a CPU 

D or MPU) of the system or apparatus from the storage 

RJ medium, and then executing the program codes. 

%9 15 In this case, the program codes read from the 

storage medium implement the novel functions of the 
invention, and the storage medium storing the program 
codes constitutes the invention. 

Further, the storage medium, such as a floppy disk, 
20 hard disk, optical disk, magneto-optical disk, CD-ROM, 
CD-R, magnetic tape, non-volatile type memory card or 
ROM can be used to provide the program codes . 

Furthermore, besides the case where the aforesaid 
functions according to the embodiments are implemented 
25 by executing the program codes read by a computer, it 

goes without saying that the present invention covers a 
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case where an operating system or the like running on 
the computer performs a part of or the entire process in 
accordance with the designation of program codes and 
implements the functions according to the embodiments. 
5 It goes without saying that the present invention 

further covers a case where, after the program codes 
read from the storage medium are written in a function 
q expansion board inserted into the computer or in a 

]P memory provided in a function extension unit connected 

£7 10 to the computer, a CPU or the like contained in the 

function expansion board or function expansion unit 
^ performs a part of or the entire process in accordance 

y with the designation of program codes and implements the 

fy function of the above embodiments . 

l0 15 Although a camera has been mentioned as an example 

of the image sensing means in the foregoing embodiments, 
this does not impose a limitation, for a device such as 
a flat-bed scanner may also be used as such means. 

Thus, in accordance with the second embodiment of 
2 0 the present invention, as described above, it is 

possible to provide an excellent operating environment, 
without giving a client an odd impression, while setting 
a plurality of scattered imaging control areas within a 
range in which imaging can be performed by image sensing 
2 5 means whose shooting direction can be changed. 

As many apparently widely different embodiments of 
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the present invention can be made without departing f 
the spirit and scope thereof, it is to be understood 
that the invention is not limited to the specific 
embodiments thereof except as defined in the appended 
claims . 



